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Abstract — In this paper, we establish robustness to noise perturbations 
of polyhedral regularization of linear inverse problems. We provide a 
sufficient condition that ensures that the polyhedral face associated to 
the true vector is equal to that of the recovered one. This criterion also 
implies that the £ 2 recovery error is proportional to the noise level for a 
range of parameter. Our criterion is expressed in terms of the hyperplanes 
supporting the faces of the unit polyhedral ball of the regularization. 
This generalizes to an arbitrary polyhedral regularization results that 
are known to hold for sparse synthesis and analysis f 1 regularization 
which are encompassed in this framework. As a byproduct, we obtain 
recovery guarantees for £°° and £ x — £ x regularization. 

I. Introduction 
A. Polyhedral Regularization 

We consider the following linear inverse problem 

y - $x +w, (1) 

where y £ R'' are the observations, xo £ M. N is the unknown true 
vector to recover, w the bounded noise, and $ a linear operator which 
maps the signal domain R^ into the observation domain R . The 
goal is to recover xq either exactly or to a good approximation. 

We call a polyhedron a subset V of R^ such that V = 
{x £ M. N | Ax < 6} for some A £ R N » xN and b £ R Nh , where 
the inequality ^ should be understood component-wise. This is 
a classical description of convex polyhedral sets in terms of the 
hyperplanes supporting their (N — 1) -dimensional faces. 

In the following, we consider polyhedral convex functions of the 
form 



X 



Jh(x) — max (x, hi 

l^i^N H 



where H 



(hi 



,N H 



»NxN H 



Thus, Vh 



{x £ R^ | Jh(x) ^1} is a polyhedron. We assume that Vh 
is a bounded polyhedron which contains in its interior. This 
amounts to saying that Jh is a gauge, or equivalently that it is 
continuous, non-negative, sublinear (i.e. convex and positively 
homogeneous), coercive, and Jh{x) > for x ^ 0. Note that it is 
in general not a norm because it needs not be symmetric. 

In order to solve the linear inverse problem ([T), we devise the 
following regularized problem 



* 1 2 

x £ arginin -\\y — $x| + \Jh(x) 



(Px(y)) 



where A > is the regularization parameter. Coercivity and convexity 
of Jh implies the set of minimizers is non-empty, convex and 
compact. 

In the noiseless case, w — 0, one usually considers the equality- 
constrained optimization problem 



x* £ argmin Jh(x) 

<fcx=y 



(My)) 



B. Relation to Sparsity and Anti-sparsity 

Examples of polyhedral regularization include the ^ 1 -norm, anal- 
ysis ^ 1 -norm and ^°°-norm. The I 1 norm reads 

N 
JH 1 (X) = WxJ! =^2\Xi\. 
i = l 

It corresponds to choosing Hi £ R iVx2 where the columns of Hi 
enumerate all possible sign patterns of length N, i.e. { — 1, 1}^. The 
corresponding regularized problem ( P\ (y) > is the popular Lasso yj| 
or Basis Pursuit DeNoising [2]. It is used for recovering sparse vec- 
tors. Analysis-type sparsity-inducing penalties are obtained through 
the (semi-)norm Jh(x) — \Lx\i, where L £ R PxJV is an analysis 
operator. This corresponds to using H — L* Hi where * stands for 
the adjoint. A popular example is the anisotropic total variation where 
L is a first-order finite difference operator. 
The (. x norm 



Jh^{x) — \x\ c 



max Xi 

l<i<iV 



corresponds to choosing H^ = [Mat, — Idjv] £ R iVx2Jv . This 
regularization, coined anti-sparse regularization, is used for instance 
for approximate nearest neighbor search [3]. 

Another possible instance of polyhedral regularization is the group 
I 1 — £°° regularization. Let B be a partition of {1, . . . ,N}. The 
I 1 — l°° norm associated to this group structure is 



Jh% (x) 



beB 



This amounts to choosing the block-diagonal matrix H^ £ 
R-^xllbee 2 \ b \ sucn ma t eac h column is chosen by taking for each 
block a position with sign ±1, others are 0. If for all b £ B, \b\ = 1, 
then we recover the ^ 1 -norm, whereas if the block structure is 
composed by one element, we get the ^°°-norm. 

C. Prior Work 

In the special case of i 1 and analysis £ 1 penalties, our criterion is 
equivalent to those defined in [4J] and [50 . To our knowledge, there is 
no generic guarantee for robustness to noise with £°° regularization, 
but 1 6] studies robustness of a sub-class of polyhedral norms obtained 
by convex relaxation of combinatorial penalties. Its notion of support 
is however completely different from ours. The work [7] studies nu- 
merically some polyhedral regularizations.In [8], the authors provide 
an homotopy-like algorithm for polyhedral regularization through 
a continuous problem coined adaptive inverse scale space method. 
The work [9] analyzes some particular polyhedral regularizations 
in a noiseless compressed sensing setting when the matrix $ is 
drawn from an appropriate random ensemble. Again in a compressed 
sensing scenario, the work of llOll studies a subset of polyhedral 



regularizations to get sharp estimates of the number of measurements 
for exact and £2 -stable recovery. 

II. Contributions 

Definition 1. We define the //-support supp H (:r) of a vector x £ R^ 
to be the set 

supp^ (2) = {i 6 {1, . - . , N H } I (a:, hi) = J H {x)} . 

This definition suggests that to recover signals with H-support 
supp H (a:), it would be reasonable to impose that $ is invertible on 
the corresponding subspace Ker H* , x y This is formalised in the 
following condition. 

Definition 2. A H-support I satisfies the restricted injectivity con- 
dition if 

Ker QnKer Hi = {0}, (Ci) 

where Hi is the matrix whose columns are those of H indexed by I. 

When it holds, we define the orthogonal projection Tj on 
$Ker#; : 



Observe that if A is chosen proportional to the noise level, then 
\\x* — X0I2 — 0(|w|2). The following proposition proves that the 
condition ICjj(7) > is almost a necessary condition to ensure 
the stability of the //-support. Its proof is omitted for obvious space 
limitation reasons. 

Proposition 1. Let xo £ R^XJO} and I its H-support such that |C7} 
holds. Let y — $a;o + w. Suppose that $/Ij / and ICr(I) < 0. If 
■^- < — then for any solution of \ P\(y) 1, we have supp ff (:ro) 7^ 
supp H {x*). 

B. Noiseless Identifiability 

When there is no noise, the following result, which is a straightfor- 
ward consequence of Theorem[T] shows that the condition 1C h{I) > 
implies signal identifiability. 

Theorem 2. Let xq £ ~R N \ {0} and I its H-support. Suppose that 
$/!/ 7^ and \Ch{I) > 0. Then the vector xq is the unique solution 



ofiPo(y)y 



Mi 



aj'&QUy 1 and 



rf 



Id -IV 



where U is (any) basis of Ker H\ . The symmetric bilinear form on 
R N induced by Tf reads 

(u, v) r ± = (u, rj-v), 

and we denote its associated quadratic form I ■ | ?,_,_. 

1 1 

Definition 3. Let I be a H-support such that jCil holds. The 

Identifiability Criterion of I is 



ICh(I) 



max min($rrf 4>ilj + Zi)i 

2,eKcrH, iel 



III. Proofs 

A. Preparatory Lemmata 

We recall the definition of the subdifferential of a convex function 
/ at the point x is the set df(x) is 

df(x) = {g e R N I f(y) > f(x) + (g, y - x)} . 

The following lemma, which is a direct consequence of the properties 
of the max function, gives the subdifferential of the regularization 
function Jh- 

Lemma 1. The subdifferential dJa at x £ K N reads 

dJ H {x) = HiT,i 

where I = supp H (a;) and E/ is the canonical simplex on W ' : 



ICh(I) — max 



r subj. to 



where I; Gl is the vector with coefficients 1, and <&i — <&Hj '* G 
R^ x ' ' where stands for the Moore— Penrose pseudo-inverse. 

ICh(I) can be computed by solving the linear program 

\iei,r^ (l>|r i L <l/ii + -.,) 

( Hizi = 0. 

A. Noise Robustness 

Our main contribution is the following result. 

Theorem 1. Let xo € R^ \ {0} and I its H-support such that $Ci\ 
holds. Let y = <&xo + W. Suppose that $/!/ 7^ and ICh(J) > 0. 
Then there exists two constants ci , Ci satisfying, 

II toll 2 Ci 

' < — where T — min Jh(xo) — (xo, hj) > 0, 

T Ci j£I c 

such that if A is chosen according to 

ci\\w\\2 < A < Tci, 
the vector x* £ M N defined by 

x* = (iHp*h + UMiU*$*(y - nhh) 
where U is any basis of Ker H } and 



{vi £R ]I] I vi >0, (vi,h) = l| 



A point x* is a minimizer of min^ f(x) if, and only if, £ 
df(x*). Thanks to Lemma[T| this gives the first-order condition for 



the problem {P\ (y) 1 



Lemma 2. A vector x* is a solution of ( P\ (y) 1 if and only if there 
exists vi £ T,i such that 

$*($£ - y) + XH1V1 = 0, 

where I = supp iif (a;). 

We now introduce the following so-called source condition. 
(SCor): For / = supp H (x), there exists r\ and vi £ £/ such that: 

$*7/ = Hivi £ 8Jh{x). 

Under the source condition, a sufficient uniqueness condition can 
be derived when Vi lives in the relative interior of £/ which is 

riE/ = tvi el 1 ' 1 I Vi > 0,{vi, h) = l|. 



Lemma 3. Let x* be a minimizer of iP\(y)\ (resp. <Po(y)i) and 
I — supp H (x*). Assume that (SC^*) is verified with Vi £ ri Ej, 



< p, — Jh(x ) + 



(<I>iI/, w) r ± - A 



I<Mi|3 



(2) 



and that \Ci\ holds. Then x* is the unique solution of 1 P\ (y) 
(resp. \Po{y)\). 



is the unique solution of 1 P\ (y) I, and x* lives on the same face as 
x , i.e. supp H (a;*) = supp H (s ). 



The proof of this lemma is omitted due to lack of space. Observe 
that in the noiseless case, if the assumptions of Lemma[3]hold at xo, 
then the latter is exactly recovered by solving iPp(y)\. 



Lemma 4. Let x* G Mr 1 and I = supp ir (a;*). Assume jCi\ holds. 
Let U be any basis of Ker Hj . There exists zi G Ker Hi such that 

U*$*($x* -y)-0 

Vi = Zi+ -Hf$*{y - $£*) G £/, 



if, and only if x* is a solution of \P\(y)\ . Moreover, if ' Vi G ri Sj 
f/jen a;* is ?Ae unique solution of \T\(y)y. 



Proof: We compute 

$*($a;* — y) + XHiVi 
=$*($i* -j/) + AH/ (z/ + i///$*(j/ - $x* 
=(Id - /////+)$* ($x* - y) = proj H . (#*(*a:* - J/)) = 0, 

where proj^. is the projection on Ker //p. Hence, x* is a solution 
of p^xjy)}. If Vi G ri£j, then according to Lemma [3] x* is the 
unique solution. ■ 

The following lemma is a simplified rewriting of the condition 
introduced in Lemma [4] 

Lemma 5. Let x* G Mr, I — supp ff (a;*) and /i = Jh(x*). 
Assume \Ci\ holds. Let U be any basis of Ker H*[. There exists 
z G Ker Hi such that 

vi = zi+ -r$iTf(y - n®ili) G Ej, 



1. Expression of x*. One has x* — fiHj*Ii + Ua where m — 
Jh(x*). Hence, 

£/*$*($a: -y) = iiU*<$>*<&H+'*I I + (U*$*$U)a - U*$*y = 0. 

Thus, 

Ua = UMiU*$*(y - n$Hp*h). 

Now, since y = $xo + W, with supp H (xo) = /, then 

x* = ixHf'*li + UMiU*^*{y - ^Hj'*li) 

= tiHJ'*li + t/M/[/*$*(( A to - v)$>Hp*h +w) + Ua 

= xo - (mo - y)HJ'*h + UMjU'&dno - /*)$#+'*!, + w), 

where Mo = Jh(xo). Hence, x* is satisfying 
x* = x + (iJ,o-IJ,)[UMiU*$*<i>-Id]Hp*h+UMiU*3>*w. (3) 

2. Checking that the //-support of x* is /. To ensure that the 
H -support of x* is / we have to impose that 

Vi G /, (hi, x*) — Jh(x*) — n 
Vj G I c , (hj, x*) < Jh{x*) = ix. 

The components on I of x* are satisfying Hix* — fjJi. Since Jh 
is subadditive, we bound the components on I c by the triangular 
inequality on 101 to get 

max{/ij, x*) ^max{hj, Xo) 



if, and only if x* is a solution of I P\ (y) i. Moreover, if Vi G ri Ej, 
then x* is the unique solution of ^P\(y)\ . 

Proof Note that any vector x G R^ such that the condition l(C7J 
holds, where / is the //-support of x, is such that 

x = fj,H[*Ii + Ua where fi = Jh(x), 

for some coefficients a and U any basis of Ker H} . We obtain 

U$*{$>x* -y) = nU&'$Hp*Ii - U$*y + U$*$Ua = 

Since \Ci\ holds, we have 

a = ([/$*Wa) _1 E/$* (y - M$A) ■ 

Hence, 

$Ua = Tj (y - fi^ilA . 

Now since, x* — nH~['*Ii + Ua, one has 



J'6i" 



Denoting 



jG/ c 



+ (mo - M)l-H"/*c[f/M 7 ?7*$*-l' - Id]//+ 

+ \\HiaUMiU*^w\\ 00 . 



■1 \V 

Subtracting y and multiplying by <3>| both sides, and replacing in the 
expression of Vi in Lemma [4] we get the desired result. ■ 

B. Proof of Theorem \l\ 

Let / be the //-support of xq. We consider the restriction 
of {P\(2/7|l to the //-support /. 



c*i = |.ar;„[i7Afii7**** - hi-h^-'IjIo, 

C 2 =\\H%UMiU*$*\\ 2>ao , 
T = no — max(/ij, ao), 

we bound the correlations outside the //-support by 

max.{hj, x*} < Mo — T + (mo — m)Ci + C2|w|. 

There exists some constants Ci,C2 satisfying ci|u;| < c^T + A such 
that 

^ /xo - T + (ii - ti)d + C 2 \\w\\ < fi (4) 

Under this condition, one has 

max.(hj, x ) < fi, 

jei c 

which proves that supp H (x*) = /. 

3. Value of n = Jh{x*). Using Lemma [5] with H = U* H, since 
x* is a solution of \V\{y)i •, there exists Zi G Ker Hi such that 



17 



z/ + -$ir^-(j/-M*/ix) e s 7 . 



(5) 



* 1 2 

2; = argmax -\\y - ^x\\ 2 + Jh{x) 

xm N z 

supPjf(i)C/ 



(Vx(y)i) 



Thanks to {Ci\ , the objective function is strongly convex on the set 
of signals of //-support I. Hence x* is uniquely defined. The proof is 
divided in five parts: We give (1.) an implicit form of x* . We check 
(2.) that the //-support of x* is the same as the //-support of xo. 
We provide (3.) the value of Jh{x*). Using Lemma[5] we prove (4.) 



that x* is the unique minimizer of {P\(y) 1 



We decompose xo as 

x = Mo-H"^'*!/ + Ua . 
Since y — $xo + W, we have 

rfy = rf(jM}$iii + ®u ao + w ). 

Now since 

TiWao = $(7(I7*$*<1'C/)~ 1 I7*$*<1'C/qo = $Uao, 



one obtains 

r i L y = /i r i L $ / ii + rf™. 

Thus, equation (O equivalently reads 



V1—Z1 + T^/Tr f (no - fj,)$ih + w 
In particular, (vi, I/) = A. Thus, 

A = {Xvi, h) = (Xzi, h) + ($Jr#((Mo - p)$ih + w, h). 
Since Zi £ Ker Hi, one has {zi, Ii) = 0. 

a = (^riii/io - /x)#jii + w, h) 

= (mo - M)|*/Ij|ra. + (4/1/, w)rf • 
Thus the value of // is given by 

($/I/, ui) r j_ - A 



(1 = 110 + 



II^I/II? 



>0. 



(6) 



4. Checking conditions of Lemma |H Consider now the vector 
vi defined by 



where 



Zi 



Vi =zi + T&*i r f ( (Mo - aO<M/ + "' 



argmax min^jTr $/I/ + Zi)i 



// — //0 \zj SKcrHj >s/ 

Under condition (|4}, the iPsupport of a;* is /, hence we only have 
to check that vi is an element of ri £/. Since {zi, I/) = 0, one has 

{vi, h) 

= {zi + -r&Ti ((/*o - n)$ih + wj , h) + (zi - zi, I/) 

={vi,h)+0 = X. 

Plugging back the expression l|6} of (p,o — p) in the definition of vi, 
one has 

1 ( ~, ± {*/!/, ™> r ^- A \ 

vi = ~zi + -\ ^iVJw + J $JIY*/I/ • 

A y 1^/i/IpjL y 

For some constant C3 such that C3|tu| — ICh(I) ■ A > 0, one has 

Vi el, Vi> 0. 

Combining this with the fact that (vi, I/) = A proves that vi £ ri £/. 
According to Lemma [5] x* is the unique minimizer of {PT^/)}. ■ 

C. Proof of Theorem \2\ 

Taking w — in Theorem [TJ we obtain immediately 

Lemma 6. Let xo £ R^ \ {0} and I its H-support such that \Ci\ 
holds. Let y = <l?xo. Suppose that $/I/ 7^ and ICh(I) > 0. Let 

T — min Jh(xo) ~ (xo, hj) > and A < Tc/. T^era, 

je/ c 



x = xo + ■ 



A 



\*Ml± 



[UMiU*<f>*<f>-ld}H? 



is the unique solution of iP\(y) 1. 

The following lemma shows that under the same condition, xo is 



a solution of iPo(y) 1. 

Lemma 7. Le? xo £ R^ \ {0} and I its H-support such that $C~i\ 
holds. Let y — $xo. Suppose that $/I/ 7^ and IC//(7) > 0. Then 



Proof: According to Lemma [6] for every < A < Tci, 
A 



X A = Xo + 



SAlJx 



[[/Mj[/*<I>*$-Id]/Y+'*I 



is the unique solution of tP\(y) 1. 

Let i/io such that $2 = y. For every < A < Tci, since a; J 



is the unique minimizer of {P\(y) 1, one has 



Using the fact that $x — y — &xo, one has Jh(xJ) < Jh(x). By 
continuity of the mapping x i-» Jh(x), taking the limit for J^O 
in the previous inequality gives 

Jh(xo) < Jh(x). 



It follows that xo is a solution of ( |-Po(y)| >. ■ 

We now prove Theorem [2] 

Proof of Theorem \2} Lemma [7] proves that xo is a solution 



of 1 Po (y) 1. We now prove that xo is in fact the unique solution. Let 
zi be the argument of the maximum in the definition of ICh{I). 
We define 

2/ + $ i T i L i>/i/ , 
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l*/I/£- 



By definition of ICh(J), for every i £ I, Vi > and («/, If) = 1. 
Thus, fl/U/ £ ri(9Jfr(xo)). Moreover, since 5/ £ KerHi, one has 

fl/«/ = HiHp*$*ri$iIi = $*?7 where 77 = T] 1 '!/!/. 



Thanks to Lemma [3] xo is the unique solution of iPq(j/)i 



xo is a solution of 1 Po (y) •■ 
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